The flexibility of models of recognition memory: The case of confidence ratings

The normalizedmaximum likelihood (NML) index is amodel-selection index derived from theminimumdescription length principle. In contrast to traditional model-selection indices, it also quantifies differences in flexibility between models related to their functional form. We present a new method for computing the NML index for models of categorical data that parameterize multinomial or productmultinomial distributions and apply it to comparing the flexibility ofmajormodels of recognitionmemory for confidence-rating based receiver-operating-characteristic (ROC) data. NML penalties are tabulated for datasets of typical sizes and interpolation functions are fitted that allow one to interpolate NML penalties for datasetswith sizes between the tabulated ones. Recovery studies suggest that the NML index performs better than traditional model-selection indices in model selection from ROC data. In an NML-basedmetaanalysis of 850 ROC datasets, versions of the dual-process signal detection models received most support followed by the finite mixture signal detection model and constrained versions of two-high threshold models. © 2015 Elsevier Inc. All rights reserved. Recognition memory has frequently been studied by means of mathematical models (for a review, see Malmberg, 2008 and Yonelinas & Parks, 2007). A range of models has been proposed. In some models, information from memory is represented in terms of discrete states; in others, a continuous representation of evidence is postulated. Discrete-state models are variants of the so-called threshold models (e.g., Blackwell, 1963 and Snodgrass & Corwin, 1988); the continuous models are variants of the socalled signal-detection models (Macmillan & Creelman, 2005). Finally, hybridmodels implement combinations of both ideas.Model fits and comparisons are frequently based on the shape of the observed receiver operating characteristic (ROC) functions. ∗ Corresponding author. E-mail address: [email protected] (K.C. Klauer). 1 Both authors contributed equally to this paper. 1. Receiver operating characteristics In the most basic recognition experiment, participants study items to be remembered later. In a subsequent test phase, they are shown the studied items mixed with new items, so-called distractors, and their task is to discriminate studied items from new items. These data are typically modeled in terms of two probabilities, the probability to respond OLD given an old item and the probability to respond OLD given a new item, also called the hit and false-alarm rates, respectively. An important line of research is to obtain hit and false-alarm rates at different levels of response bias and to plot hit rates against false alarm rates across levels of response bias resulting in a so-called ROC function. Fig. 1 shows examples of typical ROCs: From top to bottom, they are typical of an ROC generated by a threshold model, a signal-detection model with higher variability of the memory response for old items than for new items, and a simpler signal detection model with equal http://dx.doi.org/10.1016/j.jmp.2015.05.002 0022-2496/© 2015 Elsevier Inc. All rights reserved. K.C. Klauer, D. Kellen / Journal of Mathematical Psychology 67 (2015) 8–25 9 Fig. 1. Example ROC functions. variability for old and new items. All of these fall above the chancelevel diagonal that is also shown. Different levels of response bias are traditionally produced via manipulations of base rates of old relative to new items in the test phase or by payoff manipulations (Bröder & Schütz, 2009). The vast majority of studies in the field has, however, relied on a less expensive method of generating ROC data via confidence ratings (see Wixted, 2007 and Yonelinas & Parks, 2007, for reviews). That is, ratings of confidence in the OLD or NEW response, as the case may be, are collected and the different levels of confidence interpreted as expressing different levels of response bias. Although most widely used in research on recognition memory, confidence-rating data and (some of) the above models also play an important role in perception (e.g., Swets, Tanner, & Birdsall, 1961) and reasoning (e.g., Dube, Rotello, & Heit, 2010). Despite decades of research, the question which of the above models provide the best description of the data in recognition memory is still under debate (e.g., Bröder & Schütz, 2009; Dube & Rotello, 2012; Kellen & Klauer, 2014; Kellen, Klauer, & Bröder, 2013; Kellen, Singmann, Vogt, & Klauer, 2015; Onyper, Zhang, & Howard, 2010; Province & Rouder, 2012;Wixted, 2007 and Yonelinas & Parks, 2007). To our minds, several factors have prevented this very productive field from reaching a clear and non-contested decision on the most adequate model. Among these are the distorting, but often ignored influences of individual differences in memory performance and response-bias settings and of analogous differences between items (Klauer & Kellen, 2010; Rouder & Lu, 2005), an over-reliance on one method, the confidence-rating paradigm, and the absence of model-selection measures that take into account differences between models in flexibility related to functional form. The purpose of the present manuscript is to address this last problem for the important case of confidence-rating data in a similar fashion as described by Kellen et al. (2013) and Klauer and Kellen (2011a) for binary OLD/NEW ROC data (see also Kellen & Klauer, 2011). It turns out that some of the solutions developed here are evenmore widely applicable than defined by this original purpose as elaborated on below.